Integrating Paper Annotations with Electronic Environments

In this paper we discuss mechanisms by which interactions on paper can be made to affect the electronic environment. We envision a framework where there is continuous, cyclical transfer of knowledge between the paper and the electronic worlds. In this paper, we will focus on analysis of paper interaction and a framework to make changes to electronic environment. There are two key contributions: (a) We show a simple technique that detects annotations on paper using image registration techniques. (b) We show how each type of interaction – underlining, highlighting and scribbling, taking notes on paper and making changes to the media represented on paper, needs to analyzed and then incorporated into the user context model, in a meaningful way. Changes to user context will influence further interaction with the electronic world. Our preliminary user studies indicate that the framework is very promising, and useful.